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DETAILED ACTION 

Allowable Subject Matter 

1 . Claims 6,16, 26 and 34 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

2. In regards to claims 6, 16, 26 and 34 the prior art of record fails to teach or 
suggest wherein said calculating user position step detects the effects of viewing angles 
and gives higher weight to target markers that are detected at more reliable angles. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth In section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-2, 7-12, 17-22, 27-30 and 35-36 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Sauer et al. (U.S. Pub. No. 2002/0075201) in view of Rekimoto 
et al. (CyberCode: Designing Augmented Reality Environments with Visual Tags). 

5. In regards to claim 1 Sauer et al. teaches a movable head-mounted display 
(HMD) system worn by a user, wherein said HMD includes an imaging camera, tracking 
camera (video camera) and display (video display). Sauer et al. both teaches and 
illustrates providing and distributing a plurality of target markers within a workshop area 
(p. 1 , 11 1 2-1 3; p. 3, U 35, lines 1 0-1 2; p. 3, H 43; Fig. 4). Said workshop area is 
considered to read on the limitation "room." 
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Sauer et al. fails to explicitly teach wherein each of said plurality of target 
markers are distinct from all other target markers in said plurality of target markers and 
distinct from rotated versions of itself. Rekimoto et al. teaches a 2D-barcode system 
designed to be used in several augmented reality system (p. 2, column 1, 2), which 
utilizes a plurality of markers (printed tags) which are distinct from all other target 
markers in said plurality of target markers and distinct from rotated versions of itself 
(Fig. 4). 

It would have been obvious to one skilled in the art, at the time of the applicant's 
invention, to incorporate printed tags taught by Rekimoto et al. into the system taught by 
Sauer et a!., because utilizing said printed tags would provide a less expensive and 
more versatile tagging technology, which could be easily created and attached to almost 
any physical object (p. 1 , column 2, j[ 2), thus allowing for a wide application of said 
tags. 

Sauer et al. teaches that said tracking camera captures a field of view (video 
signal of a portion of said room) including a marker structure. The markers structure 
includes a plurality of light emitting diodes, visible to the tracking camera which locates 
said markers (p. 1 , H 1 3, lines 3-5; p. 1 , ^ 1 4; p. 3, |I 35, lines 1 0-1 7; Fig. 1 ). Sauer et 
al. teaches calculating user position within said room using relative position of identified 
target markers (p. 3, U 35, lines 10-17; p. 3, H 40, lines 4-7). It is noted the 
determination of the position and orientation (i.e. viewing angle) of said imaging 
cameras, through the use of said tracking camera, is considered to result in the position 
of said user which is wearing said HMD which comprises said imaging camera. 
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Sauer et al. teaches streaming 3D video content to the user through said video 
display (p, 4, U 47, lines 19-25, p. 4, T| 51; Fig. 1). It is noted that as the relationship 
between the tracking and imaging cameras assist in generating said video content that 
any adjusted position or viewing angle information set for said cameras are considered 
to be incorporate into said video content. 

6. In regards to claim 2 Sauer et al. teaches a target calibration means for 
automatically calibrating relative positions of said plurality of target markers within said 
room (p. 3, U 43). 

7. In regards to claim 7 Sauer et al. and Rekimoto et al. fail to explicitly teach 
wherein said receiving step receives said video signal via a wireless link. Official Notice 
is taken that both the concept and the advantages of transferring and receiving video 
signals via a wireless link is well known and expected in the art. It would have been 
obvious to one skilled in the art, at the time of the applicant's invention, to incorporate 
the utilization of wireless technology for the transfer and receiving of video information 
into the system taught by Sauer et al. and Rekimoto et al., because through such 
incorporation it would allow the user of said HMD more freedom of movement as they 
would be less restricted then they would be if tethered to another object by physical 
wires. 

8. In regards to claim 8 Sauer et al. and Rekimoto et al. fall to explicitly teach 
wherein said streaming step streams said 3D video content to the user through said 
video display via a wireless link. Official Notice is taken that both the concept and the 
advantages of transferring and receiving video signals via a wireless link is well known 
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and expected in the art. It would have been obvious to one skilled in the art, at the time 
of the applicant's invention, to incorporate the utilization of wireless technology for the 
transfer and receiving of video information into the system taught by Sauer et al. and 
Rekimoto et al., because through such incorporation it would allow the user of said 
HMD more freedom of movement as they would be less restricted then they would be if 
tethered to another object by physical wires. 

9. In regards to claim 9 Sauer et al. teach that the virtual view is rendered according 
to the camera pose information, determined by the processor, and blended (overlaid) 
with the corresponding video images. The augmented images can then be displayed 
stereoscopically (p. 4, 1|43, lines 5-9; Fig. 1). 

10. In regards to claim 10 it is noted said system is considered to include a sufficient 
number of target markers (Fig. 4) such that at least one target market us always visible 
in said video signal. 

11. In regards to claim 1 1 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. Sauer et al. further teaches a module 1 16 for receiving a video 
signal, a module 1 12 for identifying at least one target marker in said video signal, a 
module 1 18 for calculating user position within said room using relative positioning of 
identified target markers, a module 122 for streaming 3D video content to the user 
through said video display 124 and a module 1 18 for dynamically repositioning the 
user's perspective viewpoint within said 3D content (Fig. 1 ). It is noted said system is 
considered to perform the process. 
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12. In regards to claim 12 the rationale disclosed in the rejection of claim 2 is 
incorporated herein. 

13. In regards to claim 17 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

14. In regards to claim 18 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 

15. In regards to claim 19 the rationale disclosed in the rejection of claim 9 is 
incorporated herein. 

16. In regards to claim 20 the rationale disclosed in the rejection of claim 10 is 
incorporated herein. 

17. In regards to claim 21 the rationale disclosed in the rejection of claim 1 is 
incorporated herein. It is noted that said system is considered to perform the process, 

18. In regards to claim 22 the rationale disclosed in the rejection of claim 2 is 
incorporated herein, 

19. In regards to claim 27 the rationale disclosed in the rejection of claim 7 is 
incorporated herein. 

20. In regards to claim 28 the rationale disclosed in the rejection of claim 8 is 
incorporated herein. 

21 . In regards to claim 29 the rationale disclosed in the rejection of claim 1 1 is 
incorporated herein. 

22. In regards to claim 35 the rationale disclosed in the rejection of claim 17 is 
incorporated herein. 
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23. In regards to claim 36 the rationale disclosed in the rejection of claim 20 is 
incorporated herein. 

24. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject nnatter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

25. Claims 3-5, 13-15, 23-25 and 31-33 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Sauer et al. (U.S. Pub. No. 2002/0075201 ) and Rekimoto et al. 
(CyberCode: Designing Augmented Reality Environments with Visual Tags), as applied 
to claims 1-2, 7-12, 17-22, 27-30 and 35-36, in view of Brady et al. (U.S. Patent No. 5. 
473, 931). 

26. In regards to claim 3 Sauer et al. and Rekimoto et al. fail to explicitly teach 
identifying pairs of target markers. Brady et al. teaches identifying pairs of targets 
markers 22 (Figs. 2, 5). It would have been obvious to one skilled in the art, at the time 
of the applicant's invention, to incorporate the teachings of Brady et al. into the system 
taught by Sauer et al., because Sauer et al. teaches that calibration amongst a plurality 
of markers for the calibration of said cameras is needed for the registration between 
real objects and virtual objects, but Sauer et al. does not elaborate on the details of said 
calibration. Brady et al. provides said detail into the means of said calibration, wherein 
said calibration allows for the translation of information from one coordinate system (i.e. 
to another coordinate system (Brady et al., column 3, lines 41-46), thus allowing for said 
information to be retrieval and utilized accordingly. 
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27. In regards to claim 4 Sauer et al. teaches performing calibration in which the 
geometric relationship between marks are known (p. 3, U 43, lines 5-8). Sauer et al. 
and Rekimoto et al. fail to explicitly teach determining the position of each target in a 
target pair relative to said video camera, calculating positioning of each target in a target 
pair relative to each other and storing the relative positioning of the target pair in a list of 
relative target transforms. Brady et al. teaches determining the position of said markers 
from a video camera (column 3, line 48), calculating positioning of each target in a 
target pair relative to each other (column 3, lines 51-54) and storing the relative 
positioning of the target pair in a list (lookup table) of relative target transforms (column 
8, lines 40-50). The motivation disclosed in the rejection of claim 3 is incorporated 
herein. 

28. In regards to claim 5 Sauer et al. and Rekimoto et al. fail to explicitly teach 
determining the relative position of identified target markers using said relative target 
transforms. Brady et al. teaches utilizing said lookup table for the purposes of marker 
information retrieval (column 8, lines 40-50). The motivation disclosed in the rejection of 
claim 3 is incorporated herein. 

29. In regards to claim 13 the rationale disclosed in the rejection of claim 3 is 
- incorporated herein. 

30. In regards to claim 14 the rationale disclosed in the rejection of claim 4 is 
incorporated herein. 

31 . In regards to claim 15 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 
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32. In regards to claim 23 the rationale disclosed in the rejection of claim 3 is 
incorporated herein. 

33. In regards to claim 24 the rationale disclosed in the rejection of claim 4 is 
incorporated herein. 

34. In regards to claim 25 the rationale disclosed in the rejection of claim 5 is 
incorporated herein. 

35. In regards to claim 31 the rationale disclosed in the rejection of claim 13 is 
incorporated herein. 

36. In regards to claim 32 the rationale disclosed in the rejection of claim 14 is 
incorporated herein. 

37. In regards to claim 33 the rationale disclosed in the rejection of claim 15 is 
incorporated herein. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter-Anthony Pappas whose telephone number is 571- 
272-7646. The examiner can normally be reached on M-F 9:00am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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